10. Understand the Shapiro-Wilk's test

Understanding Normal Distribution and Shapiro-Wilk's Test

Concepts Covered:

• Normal Distribution

  • Commonly used in finance to model asset returns but not universally applicable.
  • May not capture phenomena with heavy tails, leading to potentially misleading analyses.

• Heavy or Fat Tails

  • Distributions that decline at a slower rate compared to normal distribution as they move from the center.
  • Important to identify to avoid incorrect modeling results in trading strategies.

• Shapiro-Wilk's Test

  • A statistical test to verify if data is normally distributed.
  • Test Statistic: A calculated value indicating how data fits with normal distribution expectations.
  • P-Value: Probability measure indicating how "normal" the dataset is. Small p-values suggest data may not follow a normal distribution.
  • Threshold determination is essential for decision-making (e.g., 5% threshold).

Practical Steps:

1. Use SciPy Package

   • Import Shapiro from scipy.stats to test for normality.
   • Provides outputs: test statistic and p-value.

2. Testing Process

   • Generate a normally distributed dataset.
   • Calculate test statistic and p-value to assess normality.
   • Analyze impact of transformations (e.g., squaring elements) on normality via p-value changes.